-
Notifications
You must be signed in to change notification settings - Fork 5.3k
Update OpenSUSE Helix testing from 15.6 to 16.0 #122371
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Co-authored-by: richlander <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR updates the OpenSUSE Helix testing platform version from 15.6 to 16.0 across CI pipeline configuration files. This is an infrastructure-only change that upgrades the test execution environment to use a newer OpenSUSE version.
- Updated OpenSUSE version references from 15.6 to 16.0 in Helix queue configurations
- Modified both the variable definition and the CoreCLR extra-platforms test queue
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| eng/pipelines/helix-platforms.yml | Updated the helix_linux_x64_opensuse_latest variable to reference OpenSUSE 16.0 container and updated the comment indicating the latest version |
| eng/pipelines/libraries/helix-queues-setup.yml | Updated the CoreCLR extra-platforms inner loop test queue to use OpenSUSE 16.0 instead of 15.6 |
|
/azp run runtime-extra-platforms |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
I'm seeing a failure with OpenSUSE 16.0. Looks like something very basic. Context: /root/helix/work/correlation/dotnet exec --runtimeconfig System.Net.Security.Unit.Tests.runtimeconfig.json --depsfile System.Net.Security.Unit.Tests.deps.json xunit.console.dll System.Net.Security.Unit.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
popd
===========================================================================================================
/root/helix/work/workitem/e /root/helix/work/workitem/e
Discovering: System.Net.Security.Unit.Tests (method display = ClassAndMethod, method display options = None)
Discovered: System.Net.Security.Unit.Tests (found 84 of 88 test cases)
Starting: System.Net.Security.Unit.Tests (parallel test collections = on [2 threads], stop on fail = off)
System.Net.Security.Tests.NegotiateAuthenticationTests.RemoteIdentity_ThrowsOnDisposed [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NegotiateCorrectExchangeTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmCorrectExchangeTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.Package_Unsupported_NTLM [FAIL]
Assert.Equal() Failure: Values differ
Expected: Unsupported
Actual: ContinueNeeded
Stack Trace:
/_/src/libraries/System.Net.Security/tests/UnitTests/NegotiateAuthenticationTests.cs(95,0): at System.Net.Security.Tests.NegotiateAuthenticationTests.Package_Unsupported_NTLM()
/_/src/coreclr/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.CoreCLR.cs(36,0): at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
/_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(57,0): at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
System.Net.Security.Tests.NegotiateAuthenticationTests.Package_Supported_NTLM [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmSignatureTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmIncorrectExchangeTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmEncryptionTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmIntegrityCheckTest [SKIP]
Condition(s) not met: "IsNtlmAvailable" |
|
Looks like I didn't properly test Ubuntu 26.04, at least not in all test runs, before merging. My mistake. @tarekgh Looks like very famiiar glob/loc issues for a new generation of Linux distros. We hit similar sorts of issues with Ubuntu 24.04 and Debian 13 when we were adding coverage for within the last 18 months. These issues should be resolved as part of a new PR, but this is the first I'm seeing them so registering them here and since they are easy to see here. /__w/1/s/.packages/microsoft.dotnet.helix.sdk/11.0.0-beta.25609.108/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(44,5): error : Test System.Globalization.Tests.IdnMappingIdnaConformanceTests.GetUnicode_Invalid has failed. Check the Test tab or this console log: https://helix.dot.net/api/2019-06-17/jobs/157c8fdc-0820-4618-b581-23249eb59e46/workitems/System.Globalization.Extensions.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/11.0.0-beta.25609.108/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(44,5): error : Test System.Globalization.Tests.IdnMappingIdnaConformanceTests.GetAscii_Invalid has failed. Check the Test tab or this console log: https://helix.dot.net/api/2019-06-17/jobs/157c8fdc-0820-4618-b581-23249eb59e46/workitems/System.Globalization.Extensions.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/11.0.0-beta.25609.108/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(44,5): error : Test System.Globalization.Tests.JapaneseCalendarTests.IsLeapMonth_Invalid_ThrowsArgumentOutOfRangeException has failed. Check the Test tab or this console log: https://helix.dot.net/api/2019-06-17/jobs/157c8fdc-0820-4618-b581-23249eb59e46/workitems/System.Globalization.Calendars.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/11.0.0-beta.25609.108/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(44,5): error : Test System.Globalization.Tests.JapaneseCalendarTests.GetMonthsInYear_Invalid_ThrowsArgumentOutOfRangeException has failed. Check the Test tab or this console log: https://helix.dot.net/api/2019-06-17/jobs/157c8fdc-0820-4618-b581-23249eb59e46/workitems/System.Globalization.Calendars.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/11.0.0-beta.25609.108/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(44,5): error : Test System.Globalization.Tests.JapaneseCalendarTests.GetLeapMonth_Invalid_ThrowsArgumentOutOfRangeException has failed. Check the Test tab or this console log: https://helix.dot.net/api/2019-06-17/jobs/157c8fdc-0820-4618-b581-23249eb59e46/workitems/System.Globalization.Calendars.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/11.0.0-beta.25609.108/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(44,5): error : Test System.Globalization.Tests.JapaneseCalendarTests.ToDateTime_Invalid_ThrowsArgumentOutOfRangeException has failed. Check the Test tab or this console log: https://helix.dot.net/api/2019-06-17/jobs/157c8fdc-0820-4618-b581-23249eb59e46/workitems/System.Globalization.Calendars.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/11.0.0-beta.25609.108/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(44,5): error : Test System.Globalization.Tests.JapaneseCalendarTests.ToFourDigitYear_Invalid_ThrowsArgumentOutOfRangeException has failed. Check the Test tab or this console log: https://helix.dot.net/api/2019-06-17/jobs/157c8fdc-0820-4618-b581-23249eb59e46/workitems/System.Globalization.Calendars.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/11.0.0-beta.25609.108/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(44,5): error : Test System.Globalization.Tests.JapaneseCalendarTests.GetDaysInYear_Invalid_ThrowsArgumentOutOfRangeException has failed. Check the Test tab or this console log: https://helix.dot.net/api/2019-06-17/jobs/157c8fdc-0820-4618-b581-23249eb59e46/workitems/System.Globalization.Calendars.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/11.0.0-beta.25609.108/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(44,5): error : Test System.Globalization.Tests.JapaneseCalendarTests.IsLeapDay_Invalid_ThrowsArgumentOutOfRangeException has failed. Check the Test tab or this console log: https://helix.dot.net/api/2019-06-17/jobs/157c8fdc-0820-4618-b581-23249eb59e46/workitems/System.Globalization.Calendars.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/11.0.0-beta.25609.108/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(44,5): error : Test System.Globalization.Tests.JapaneseCalendarTests.IsLeapYear_Invalid_ThrowsArgumentOutOfRangeException has failed. Check the Test tab or this console log: https://helix.dot.net/api/2019-06-17/jobs/157c8fdc-0820-4618-b581-23249eb59e46/workitems/System.Globalization.Calendars.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj]
/__w/1/s/.packages/microsoft.dotnet.helix.sdk/11.0.0-beta.25609.108/tools/azure-pipelines/AzurePipelines.MultiQueue.targets(44,5): error : Test System.Globalization.Tests.JapaneseCalendarTests.GetDaysInMonth_Invalid_ThrowsArgumentOutOfRangeException has failed. Check the Test tab or this console log: https://helix.dot.net/api/2019-06-17/jobs/157c8fdc-0820-4618-b581-23249eb59e46/workitems/System.Globalization.Calendars.Tests/console [/__w/1/s/src/libraries/sendtohelixhelp.proj] |
|
@richlander I’m currently out and won’t be responsive, but I’ve created PR #122480 to address the failures you’re seeing. If needed, feel free to merge it before merging yours. @jeffhandley The change I made here is effectively a breaking change, and I’ve addressed the issue in .NET 11. I expect that anyone using .NET 10 on Ubuntu 26.04, or any distro with ICU v87, may run into a similar issue with the Japanese calendar. The breaking change affects the minimum supported date for the Japanese calendar, so the impact should be minimal, but we should keep an eye out for any scenarios where users might encounter problems. If necessary, we can consider porting the fix to .NET 10. |
|
Thanks @tarekgh. Much appreciated. I'm sorry that I didn't notice this issue earlier. Happy holidays. |
|
I didn't notice this Linux Arm64 failure earlier. ushd .
/root/helix/work/correlation/dotnet exec --runtimeconfig System.Text.Json.SourceGeneration.Roslyn4.4.Unit.Tests.runtimeconfig.json --depsfile System.Text.Json.SourceGeneration.Roslyn4.4.Unit.Tests.deps.json xunit.console.dll System.Text.Json.SourceGeneration.Roslyn4.4.Unit.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
popd
===========================================================================================================
/root/helix/work/workitem/e /root/helix/work/workitem/e
Discovering: System.Text.Json.SourceGeneration.Roslyn4.4.Unit.Tests (method display = ClassAndMethod, method display options = None)
Discovered: System.Text.Json.SourceGeneration.Roslyn4.4.Unit.Tests (found 60 test cases)
Starting: System.Text.Json.SourceGeneration.Roslyn4.4.Unit.Tests (parallel test collections = on [2 threads], stop on fail = off)
./RunTests.sh: line 175: 62 Killed "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Text.Json.SourceGeneration.Roslyn4.4.Unit.Tests.runtimeconfig.json --depsfile System.Text.Json.SourceGeneration.Roslyn4.4.Unit.Tests.deps.json xunit.console.dll System.Text.Json.SourceGeneration.Roslyn4.4.Unit.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e@jeffhandley -- Who should look at this? It seems to only fail on Ubuntu 26.04. |
|
/azp run runtime-extra-platforms |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
Issue is still there. /root/helix/work/workitem/e /root/helix/work/workitem/e
Discovering: System.Net.Security.Unit.Tests (method display = ClassAndMethod, method display options = None)
Discovered: System.Net.Security.Unit.Tests (found 84 of 88 test cases)
Starting: System.Net.Security.Unit.Tests (parallel test collections = on [2 threads], stop on fail = off)
System.Net.Security.Tests.NegotiateAuthenticationTests.RemoteIdentity_ThrowsOnDisposed [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NegotiateCorrectExchangeTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmCorrectExchangeTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.Package_Unsupported_NTLM [FAIL]
Assert.Equal() Failure: Values differ
Expected: Unsupported
Actual: ContinueNeeded
Stack Trace:
/_/src/libraries/System.Net.Security/tests/UnitTests/NegotiateAuthenticationTests.cs(95,0): at System.Net.Security.Tests.NegotiateAuthenticationTests.Package_Unsupported_NTLM()
/_/src/coreclr/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.CoreCLR.cs(36,0): at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
/_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(57,0): at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
System.Net.Security.Tests.NegotiateAuthenticationTests.Package_Supported_NTLM [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmSignatureTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmIncorrectExchangeTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmEncryptionTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmIntegrityCheckTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
Finished: System.Net.Security.Unit.Tests
=== TEST EXECUTION SUMMARY ===
System.Net.Security.Unit.Tests Total: 103, Errors: 0, Failed: 1, Skipped: 8, Time: 4.583s |
That test is conditional on I think best action at this point is to disable tests same way as we did for #111639 (and perhaps lump it to same tracking issue) I can look into next week if nobody else gets to it faster. |
|
Thanks for looking. I'll take whatever you think is the best course of action. |
|
/azp run runtime-extra-platforms |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
we can add OpenSUSE 16 once #123055 is merged. I would specifically add just 16 as major version, hoping 17 would pick fixed gssntlmssp-1.3+ |
|
/azp run runtime-extra-platforms |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run runtime-extra-platforms |
|
Azure Pipelines successfully started running 1 pipeline(s). |
jeffhandley
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@richlander This looks good to me. Build Analysis is saying there are known infrastructure errors, known build errors, and known test errors. Then there are extra-platforms build and test failures on android and Apple platforms that are unrelated. I recommend using /ba-g All failures are known or unrelated extra platforms errors to mark build analysis as green.
|
I still see the same networking failure on OpenSUSE 16.0 (although line numbers now differ). I validated that the merge includes the commit with the networking fix. Console log: 'System.Net.Security.Unit.Tests' from job 4675da3d-4b30-429f-9ea4-e123e3d4b06e (azurelinux.3.amd64.open.rt) using docker image mcr.microsoft.com/dotnet-buildtools/prereqs:opensuse-16.0-helix-amd64 on a007MAB
running $HELIX_CORRELATION_PAYLOAD/scripts/2efa486e4ea04e5d9db4d10fab31de43/execute.sh in /datadisks/disk1/work/AA7B0970/w/9BDF08D5/e max 900 seconds
===========================================================================================================
/root/helix/work/workitem/e /root/helix/work/workitem/e
Discovering: System.Net.Security.Unit.Tests (method display = ClassAndMethod, method display options = None)
Discovered: System.Net.Security.Unit.Tests (found 84 of 88 test cases)
Starting: System.Net.Security.Unit.Tests (parallel test collections = on [2 threads], stop on fail = off)
System.Net.Security.Tests.NegotiateAuthenticationTests.RemoteIdentity_ThrowsOnDisposed [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NegotiateCorrectExchangeTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmCorrectExchangeTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.Package_Unsupported_NTLM [FAIL]
Assert.Equal() Failure: Values differ
Expected: Unsupported
Actual: ContinueNeeded
Stack Trace:
/_/src/libraries/System.Net.Security/tests/UnitTests/NegotiateAuthenticationTests.cs(104,0): at System.Net.Security.Tests.NegotiateAuthenticationTests.Package_Unsupported_NTLM()
/_/src/coreclr/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.CoreCLR.cs(36,0): at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
/_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(57,0): at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
System.Net.Security.Tests.NegotiateAuthenticationTests.Package_Supported_NTLM [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmSignatureTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmIncorrectExchangeTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmEncryptionTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
System.Net.Security.Tests.NegotiateAuthenticationTests.NtlmIntegrityCheckTest [SKIP]
Condition(s) not met: "IsNtlmAvailable"
Finished: System.Net.Security.Unit.Tests |
that make sense. The change I did only fliped the Ubuntu. So we would need to add OpenSUSE 16 to the "UseManaged" camp. If that is extra platform I would be fine if you just merge it and we fix it afterwards. |
wfurt
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
|
I'm in no rush to merge. I'd rather we don't add new errors to |
|
What are next steps @wfurt? |
Fixes Issue
main PR
Description
Updates OpenSUSE test platform version from 15.6 to 16.0 in Helix queue configurations:
eng/pipelines/helix-platforms.yml: Updatedhelix_linux_x64_opensuse_latestvariableeng/pipelines/libraries/helix-queues-setup.yml: Updated extra-platforms CoreCLR queue definitionBoth files now reference
opensuse-16.0-helix-amd64container images andopenSUSE.16.0.Amd64.Openqueue names.Customer Impact
None. Infrastructure-only change affecting CI test execution environment.
Regression
No. This is a proactive update to use a newer OpenSUSE version for testing.
Testing
Configuration change only. Will be validated through CI pipeline execution on the updated platform.
Risk
Low. Standard platform version update. Helix infrastructure team maintains the referenced container images, and the queue naming convention follows established patterns used for other platform updates.
Package authoring no longer needed in .NET 9
IMPORTANT: Starting with .NET 9, you no longer need to edit a NuGet package's csproj to enable building and bump the version.
Keep in mind that we still need package authoring in .NET 8 and older versions.
Original prompt
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.